The Forgetron: A Kernel-Based Perceptron on a Fixed Budget
نویسندگان
چکیده
The Perceptron algorithm, despite its simplicity, often performs well on online classification tasks. The Perceptron becomes especially effective when it is used in conjunction with kernels. However, a common difficulty encountered when implementing kernel-based online algorithms is the amount of memory required to store the online hypothesis, which may grow unboundedly. In this paper we present and analyze the Forgetron algorithm for kernel-based online learning on a fixed memory budget. To our knowledge, this is the first online learning algorithm which, on one hand, maintains a strict limit on the number of examples it stores while, on the other hand, entertains a relative mistake bound. In addition to the formal results, we also present experiments with real datasets which underscore the merits of our approach.
منابع مشابه
The Forgetron: A Kernel-Based Perceptron on a Budget
The Perceptron algorithm, despite its simplicity, often performs well in online classification tasks. The Perceptron becomes especially effective when it is used in conjunction with kernel functions. However, a common difficulty encountered when implementing kernel-based online algorithms is the amount of memory required to store the online hypothesis, which may grow unboundedly as the algorith...
متن کاملMemory Constraint Online Multitask Classification
We investigate online kernel algorithms which simultaneously process multiple classification tasks while a fixed constraint is imposed on the size of their active sets. We focus in particular on the design of algorithms that can efficiently deal with problems where the number of tasks is extremely high and the task data are large scale. Two new projectionbased algorithms are introduced to effic...
متن کاملTwin Vector Machines for Online Learning on a Budget
This paper proposes Twin Vector Machine (TVM), a constant space and sublinear time Support Vector Machine (SVM) algorithm for online learning. TVM achieves its favorable scaling by maintaining only a fixed number of examples, called the twin vectors, and their associated information in memory during training. In addition, TVM guarantees that Kuhn-Tucker conditions are satisfied on all twin vect...
متن کاملOnline Learning for URL classification
This work consists of a review of online algorithms for URL classification followed by some extensions and tweaking of these methods to make them efficient in terms of computational time and memory. We found out that the trade-off in error for these extensions are fairly comparable for some of these algorithms. We also applied two more kernel based methods namely Forgetron and Projectron.
متن کاملOnline training on a budget of support vector machines using twin prototypes
This paper proposes twin prototype support vector machine (TVM), a constant space and sublinear time support vector machine (SVM) algorithm for online learning. TVM achieves its favorable scaling by memorizing only a fixed-size data summary in the form of example prototypes and their associated information during training. In addition, TVM guarantees that the optimal SVM solution is maintained ...
متن کامل